A joint introduction to Gaussian Processes and Relevance Vector Machines with connections to Kalman filtering and other kernel smoothers

The expressive power of Bayesian kernel-based methods has led them to become an important tool across many different facets artificial intelligence, and useful a plethora modern application domains, providing both interpretability via uncertainty analysis. This article introduces discusses two which straddle the areas probabilistic schemes kernel for regression: Gaussian Processes Relevance Vector Machines. Our focus is on developing common framework with view these methods, intermediate version well-known ridge regression, drawing connections among them, dual formulations, discussion their in context major tasks: smoothing, interpolation, filtering. Overall, we provide understanding mathematical concepts behind models, summarize discuss depth interpretations highlight relationship other such as linear smoothers, Kalman filtering Fourier approximations. Throughout, numerous figures promote understanding, make recommendations practitioners. Benefits drawbacks techniques are highlighted. To our knowledge, this most in-depth study its kind date focused will be relevant theoretical practitioners throughout domains data-science, signal processing, machine learning, intelligence general.